On the Implementation of Programing Languages with Neural Nets

In this paper we show that programming languages are implementable on neural nets, namely, neural nets can be designed to solve any (computable) high level programming task. Constructions like the one that follows can also be used to built large scale neural nets that integrate learning and control structures. We use a very simple model of analog recurrent neural nets and a number-theoretic approach. Our results can be generalised to other models of neural computation, and also to support structured data types (through coding techniques). The use of such a model for computability analysis is due to Hava Siegelmann. In [Siegelmann and Sontag 92, Siegelmann and Sontag 95] Hava Siegelmann and Eduardo Sontag used it to establish lower bounds on the computational power of analog recurrent neural nets. An analog recurrent neural net is a dynamic system with application map of the form x → (t+1) = φ(x → (t) , u → (t)) (1) where x i (t) denotes de activity (firing frequency) of neuron i at time t within a population of N interconnected neurons, and u i (t) the input bit of input stream i at time t within a set of M input channels. The application map φ is taken as a composition of an affine map with a picewise linear map of the interval [0,1], known as the saturated sigmoid: σ(x) =      0 if x<0 x if 0≤x≤1 1 if x>1 (2)